converge servers and io controllers with pcie n create cost-efective high-availability hyperscale systems by enabling communication between in-rack hosts and endpoints using pcie n simplify connectivity while providing the highest pcie switching performance available for data center servers, storage, and networks n reduce latency, system complexity, and power consumption by up to 50% in data-intensive environments n take advantage of industry-frst features for most demanding hyper-converged, nvme and rack scale systems avago pex9700 switches allow customers to build high performance, low latency, scalable, cost-efective pci express-based fabrics. the switches enable i/o sharing with standard sr-iov or multifunction capability, allowing multiple hosts or nodes to reside on a single pcie-based network. hosts communicate through ethernet-like dma (nic dma) with other hosts and end-points using application software. hosts may also communicate using tunneled window connection (twc), a special low latency host-to-host communication capability for short packets. shared i/o using standards pex9700 switches allow the virtual functions (vfs) of sriov endpoints (such as an avago megaraid? sas controller) to be shared and assigned to multiple hosts concurrently. each host can enumerate its assigned functions using standard bios and os software and use them with unmodifed vendor-supplied drivers. the use of standard system software minimizes software support costs. general purpose host-to-host dma ethernet is used almost universally for server to server communications. pex9700 switches contain a virtual ethernet nic at each host port that allows ethernet to be tunneled transparently through the fabric to any and all servers that are connected. internal ethernet communications using virtual ethernet nics and nic dma are complemented by the ability to share a physical sriov nic, thus providing compatibility with the vast library of applications that leverage ethernet communications. software-defned fabric the switches are built on a hybrid hardware/software platform that ofers high confgurability and fexibility in regards to the number of hosts, end-points, and pcie slots. the critical pathways have direct hardware support, enabling the fabric to ofer non-blocking, line speed performance with features such as i/o sharing and dma. the solution is completed by management processor that communicates with platform management via api and/or cli. the solution ofers an innovative approach to setup and control, making use of an of-chip management cpu (mcpu) to initialize the pex9700 switch, confgure the routing tables, handle errors, hot-plug events, and enable the solution to extend the capabilities without modifying the system software. general features n state-of-the-art switch fabric C sharing i/os among multiple hosts C host-to-host dma C low latency twc n any port can be a host port or downstream (device) port n works with standard pcie end-points and hosts C and software, as well as with existing application software n msi-x support n allows fexible fabric topologies key advantages n pci express switches C 12 to 97 lanes with integrated on-chip serdes C 5 to 25 independent ports C designate any port as the upstream port C low-power serdes (under 90 mw per lane) C device-specifc relaxed ordering C port confguration C dedicated management port for mcpu C x4, x8, or x16, depending on port confguration; x4 can down-train to x1 and x2 width C confgurable through serial eeprom, i2c, smbus, and/or host port n standards compliant C pci express base specifcation, r3.1 (backward compatible w/ pcie r2.0, & r1.0a/1.1) C pci power management spec, r1.2 n high performance C full line rate on all ports C cut-thru packet latency of less than 150ns (x16 to x16) C 2kb max payload size C multicast through dma pex9700 series switch chips managed pci express switches based on expressfabric? technology product brief
| 2 tunneled window connection (twc) the dma or twc approaches are two ways hosts can communicate. twc allows short messages to be sent from one host to another in a very low latency manner, and without the overhead associated with dma. downstream port containment (dpc/edpc) most servers have difculty handling serious errors, especially when a pcie end-point disappears from the system. dpc/edpc allows a downstream link to be disabled after an uncorrectable error, making recovery possible in a controlled and robust manner. flexible topologies pex9700 switches eliminate the topology restrictions of pcie. the switch allows other topologies such as mesh, i/o expansion box with multiple hosts, and many others. and it does this while allowing the components to remain architecturally and software compatible with standard pcie. improved ssc isolation the switches ofer several mechanisms for supporting multi-clock domains that include spread spectrum clocking; eliminating the need to pass a common clock across a backplane. in addition to the standard avago approach to the problem, a new pci-sig approach called sris (separate refclk independent ssc architecture) is now available. applications products based on expressfabric technology can help deliver an outstanding solution for designing a heterogeneous system with a requirement for a fexible mix of processors, storage elements, and communication devices. hpc clusters hpc clusters are made up of high-performance processing elements that communicate through high bandwidth, low latency pathways in order to execute applications such as medical imaging, fnancial trading, data warehousing, etc. pex9700 switches can be used in switch fabric applications for hpc clustering. the processing subsystems can be connected to the pcie fabric while running the same application software. pcie switch based clustering eliminates expensive protocol bridging devices resulting in lower cost and power. and clustering systems can be built with i/o sharing as an additional native capability when needed. software development kit (sdk) the sdk for the pex9700 series includes drivers, source code and gui interfaces to aid in confguring and debugging. both the performancepak? and visionpak? are exclusive to avago and are supported by its rdk and sdk, which are the industrys most advanced hardware-and software development kits. performancepak the performancepak is a suite of unique and innovative performance features that allows avago gen 3 switches to be the highest performing switches in the market today. visionpak the visionpak is a debug diagnostics suite of integrated hardware and software instruments that allows users to help bring their systems to market faster. key advantages (continued) n quality of service (qos) C 8 trafc classes (tc) supported n reliability, availability, serviceability C visionpak? C performance pak? C dpc/edpc support C read tracking for surprise removal C all ports hot-plug capable thru i2c Cssc isolation on all ports C sris support C ecrc and poison bit support C port status bits and gpio available product brief
pex9700 series part number lanes ports latency (ns) hpc* aggregate bandwidth ssc* dedicated x1 mcpu port dma multicast package size (mm 2 ) typical power modes power typ. (w) peer-to-peer fanout fabric PEX9797 97 25 150 6 1536gt (8.0 gt/s/lane x 96 serdes x2 (full-duplex)) 24 yes yes 35x35 23.9 24.3 20.6 25.0 pex9781 81 21 150 5 1280gt (8.0 gt/s/lane x 80 serdes x2 (full-duplex)) 20 yes yes 35x35 21.5 22.5 19.6 23.3 pex9765 65 17 150 4 1024gt (8.0 gt/s/lane x 64 serdes x2 (full-duplex)) 16 yes yes 35x35 15.9 16.2 13.9 16.9 pex9749 49 13 150 4 768gt (8.0 gt/s/lane x 48 serdes x2 (full-duplex)) 12 yes yes 27x27 13.5 14.5 12.8 15.2 pex9733 33 9 150 2 512gt (8.0 gt/s/lane x 32 serdes x2 (full-duplex)) 8 yes yes 27x27 7.9 8.1 7.2 8.9 pex9716 16 5 154 1 256gt (8.0 gt/s/lane x 16 serdes x2 (full-duplex)) 4 no no 19x19 4.0 4.0 3.8 4.8 pex9712 12 5 158 1 192gt (8.0 gt/s/lane x 12 serdes x2 (full-duplex)) 4 no no 19x19 3.5 3.7 3.4 4.4 product ordering information switch part numbers description rapid development kit (rdk) part number PEX9797-aa80bc g 97-lane, 25-port expressfabric device (35 35 mm2) PEX9797-aardk pex9781-aa80bc g 81-lane, 21-port expressfabric device (35 35 mm2) PEX9797-aardk pex9765-aa80bc g 65-lane, 17-port expressfabric device (35 35 mm2) PEX9797-aardk pex9749-aa80bc g 49-lane, 13-port expressfabric device (27 27 mm2) pex9749-aardk pex9733-aa80bc g 33-lane, 9-port expressfabric device (27 27 mm2) pex9749-aardk pex9716-aa80bc g 16-lane, 5-port expressfabric device (19 19 mm2) pex9716-aardk pex9712-aa80bc g 12-lane, 5-port expressfabric device (19 19 mm2) pex9716-aardk pxf55033-aa 32-port expressfabric top-of-rack switch box with qsfp+ connections pxf51003-aa 2-port pcie bus extender card with redrivers and qsfp+ connections acronym guide dma ............................................................................. direct memory access hpc .............................................................................. hot-plug controllers twc ............................................................................. tunneled window connection ssc ............................................................................... spread spectrum clock isolation msi-x ........................................................................... message signaled interrupts sris .............................................................................. separate refclk independent ssc architecture dpc .............................................................................. downstream port containment edpc ............................................................................ enhanced dpc commercial temperature range ...................... 0+70 (celsius) for more information, visit www.avagotech.com avago, avago technologies, the a logo, expressfabric, performancepak, and visionpak are trademarks of avago technologies in the united states and other countries. all other brand and product names may be trademarks of their respective companies. copyright ?2015 avago technologies. all rights reserved. > 06.11.15 av00-0327en product brief
|
